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ABSTRACT 

Western  Pacific  tropical  cyclone  position  forecast  errors  for  10 
years  (1966-1975)  are  statistically  analyzed.  Variations  of  errors 
versus  a dozen  parameters  are  examined  and  the  trends  over  the  10  years 
are  discussed.  Discriminant  analysis  techniques  were  used  to  isolate 
categories  where  forecasts  were  likely  to  be  above  and  below  the  median 
in  East-West  and  North-South  error  components . The  discriminant  analysis 
was  tested  on  1976  data  and  the  results  are  presented.  It  was  confirmed 
that  a small  number  of  readily  available  parameters,  such  as  location, 
maximum  winds,  and  speed  of  movement,  can,  with  reasonable  effectiveness, 
classify  a tropical  cyclone  forecast  as  representing  a group  either 
markedly  above  or  below  average  errors. 
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I .  OBJECTIVES 


r. 


As  discussed  in  a request  from  the  Director,  Joint  Typhoon  Warning 
Center,  Guam  (JTWC)  to  the  Naval  Environmental  Prediction  Research 
Facility,  Monterey  (NEPRF) , a need  exists  for  a statistical  analysis  of 
the  JTWC  tropical  cyclone  forecasts  to  discover  the  existence  of  any 
significant  trends.  More  specifically,  the  long  range  goals  number 
four: 

1.  To  identify  situations  where  the  forecasts  are  very  good  or  very 
bad,  to  allow  maximum  concentration  of  resources  for  quick  reduction  of 
the  largest  errors. 

2.  To  provide  probability  algorithms  for  an  estimate  of  the  fore- 
cast errors  of  warnings,  to  assist  Western  Pacific  commanders  in  opera- 
tional decisions  regarding  the  protection  and/or  evacuation  of  militcry 
resources . 

3.  To  stratify  errors  for  24-  through  72-hour  forecasts,  based  on 
various  parameters  such  as  location,  time  of  year,  speed  of  movement, 
intensity,  and  synoptic  patterns. 

4.  To  determine  if  the  year  to  year  variations  in  forecast  accura- 
cies for  the  10  year  period  are  real,  or  random  deviations  about  a long 
term  mean. 

The  more  immediate  short  range  goals  of  this  research,  as  a first 
step  toward  the  realization  of  the  long  range  objectives,  are: 

1.  To  check  the  data  for  errors  in  recording,  and  to  test  it  for 
reliability  as  a data  base  for  statistical  study. 
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2.  To  assemble  and  consolidate  the  data  into  a usable  format. 

3.  To  determine  basic  statistical  relationships  between  parameters. 

4.  To  manipulate  the  basic  data,  to  create  a set  of  parameters  for 
further  study  of  errors  in  the  24-,  48-,  and  72-hour  typhoon  forecasts. 

5.  To  perform  discriminant  and  stepwise  multiple  linear  regression 
analyses,  to  find  parameters  related  to  the  forecast  errors. 

6.  To  summarize  the  results  and  test  them  by  a preliminary  applica- 
tion to  1976  data. 

7.  To  make  recommendations  as  to  the  direction  for  continued 
research  toward  the  long  range  goals. 


# 

II.  DESCRIPTION  OF  THE  DATA 

The  JTWC  Western  Pacific  tropical  cyclone  forecasts  and  best  tracks 
were  matched  by  date/time  groups  for  10  years  as  follows: 

1966  - 34  storms 

1967  - 35 

1968  - 26 

1969  - 23 

1970  - 26 

1971  - 35 

1972  - 31 

1973  - 21 

1974  - 34 

1975  - 25  storms 

The  term  "storm"  used  herein  refers  collectively  to  tropical  cyclones 
(tropical  depressions,  tropical  storms,  and  typhoons)  without  regard  to 
intensity.  The  10  year  total  was  290  storms,  or  an  average  of  29  storms 
per  year.  The  total  number  of  best  track  positions  at  six-hourly 
intervals  was  approximately  6150. 

In  the  process  of  matching  forecasts  with  best  tracks  of  the  same 
time,  the  data  was  checked  for  errors  and  corrected,  when  necessary,  us- 
ing annual  typhoon  reports.  Rarely  was  a report  garbled  beyond  correc- 
tion so  as  to  require  removal  from  the  data.  There  were  some  storms, 
however,  that  were  so  short-lived  as  to  provide  no  verifying  forecasts. 
These  were  not  represented  in  the  verified  case  data  that  was  statisti- 
cally analyzed.  For  instance,  to  verify  a 48-hour  forecast  and  compute 
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an  error  distance  for  the  forecast,  a best  track  must  be  available  48 
hours  later.  If,  in  that  48-hour  period,  the  storm  dissipated  and  was 
no  longer  identified  by  a best  track  position,  the  forecast  could  not 
be  verified.  This  also  accounts  for  the  fact  that  fewer  cases  were 
verified  for  72-hour  forecasts  than  for  48-hour,  and  fewer  48-hour  than 
24-hour  forecasts.  These  cases  numbered  as  follows: 

4809  24-hour  forecasts 
3038  48-hour  forecasts 
1372  72-hour  forecasts 

As  a minimum  the  following  parameters  were  known  for  each  case  at 


forecast  initiation  time: 

1.  Maximum  Wind  MAX  WIND 

2.  Latitude  LAT 

3 . Longitude  LONG 

4.  West-East  Component  of  Storm  Movement  U MOVT 

5.  South-North  Component  of  Storm  Movement  V MOVT 

6.  Position  Number  on  Storm  Track  POS  NO 

7.  Number  of  Storms  in  Progress  at  Forecast  NO  STM 

Time 

8 . Month  MONTH 

9.  Time-GMT  of  the  Forecast  TIME 

10.  Error  Distance  (Nautical  Miles)  ERR  DIS 

11.  Direction  from  Verifying  Position  to  ERR  DIR 

Forecast  Position 


The  1976  data  was  processed  in  a similar  way,  but  retained  separately 
for  testing.  There  were  625  best  track  positions  at  six-hourly  intervals 
from  25  storms  and  the  verifying  cases  totaled: 


524  24-hour  forecasts 
424  48-hour  forecasts 
332  72-hour  forecasts 
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III.  STATISTICAL  ANALYSIS 


A.  BASIC  STATISTICS 

As  a preliminary  consideration,  evidence  of  a climatic  tr°nd  in  the 
10  year  data  was  investigated.  The  number  of  occurrences  of  tropical 
cyclones  in  a fixed  time  (1  year)  can  be  assumed  to  follow  a Poisson  dis- 
tribution if  two  plausible  conditions  exist:  (a)  an  occurrence  is  just 
as  likely  in  one  interval  as  another,  and  (b)  the  occurrence  of  an  event 
has  no  effect  on  whether  or  not  another  occurs.  A property  of  this 
distribution  requires  the  population  variance  to  equal  the  population 
mean.  In  this  10  year  sample,  the  variance  is  29.89,  and  the  mean  is 
29.00.  If  a climatic  change  were  occurring,  then  the  sample  variance 
should  exceed  the  sample  mean.  As  it  does  not,  no  climatic  trend  is 
evident  within  the  10  year  sample. 

The  initial  statistical  analysis  of  the  variables  employed  the  UCLA 
Biomedical  computer  program  BMD02R  stepwise  multiple  linear  regression 
(Dixon,  1970).  Tables  I and  II  summarize  the  means,  standard  deviations, 
variance  explained,  and  the  correlation  matrix  of  the  first  10 
variables  for  the  24-,  48-  and  72-hour  forecasts.  No  correlation  coef- 
ficient of  any  available  predictor  with  the  magnitude  of  the  errors 
(at  either  24,  48,  or  72  hours)  exceeded  0.185,  and  the  total  explained 
variance  of  the  error  distance  did  not  exceed  11%.  It  was  noted,  however, 
• that  the  variables  contributing  most  to  the  explained  variance  were 

MAX  WND,  LAT,  LONG,  U MOVT,  AND  V MOVT.  The  concept  of  predicting  the 
error  directly  was  abandoned. 
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From  these  basic  statistics,  the  average  Western  Pacific  tropical 
cyclone  moved  from  19°N  latitude,  135°E  longitude  to  the  WWW  at  seven 
knots  with  maximum  winds  of  about  66  knots  in  late  August. 

B.  MEAN  ERROR  STRATIFICATIONS 

For  more  detail,  the  forecast  errors  were  stratified  and  mean  errors 
were  computed  for  each  stratification  along  the  range  of  each  variable. 
The  distance  to  the  nearest  storm  (STM  DIS) , for  multiple  storm  cases, 
and  initial  position  error  (POS  ERR)  were  added  as  variables.  Signifi- 
cant trends  were  evident  (Figures  1-6).  In  the  figures,  stratifications 
were  selected  to  keep  the  number  of  cases  in  each  group  relatively  high. 
Frequencies  are  typically  a few  hundred  and  are  not  indicated  except 
where  they  drop  below  100.  As  an  aid  in  interpreting  these  figures, 
relative  frequencies,  all  based  on  4809  cases,  are  shown  in  Figures  1-6 
for  the  24-hour  forecasts.  Since  there  are  in  percentages  of  the  total, 
they  roughly  apply  also  for  48-  and  72-hour  forecasts. 

In  each  of  the  24-,  48-,  and  72-hour  forecast  situations,  the  mean 
forecast  errors  were  minimal  for  lower  latitudes  (Figure  la) , gradually 
increasing  with  latitude.  This  indicates  that  storms  cure  more  accurately 
forecast  before  they  recurve  ar.d  move  into  higher  latitudes.  It  is 
als'  consistent  with  a study  by  Sadler  (1967) , which  distinguished 
between  (a)  storms  originating  in  the  vicinity  of  the  mean  August  surface 
trough,  between  5°  and  20°N  latitude,  that  moved  mostly  to  the  west;  and 
(b)  those  beginning  north  of  20°  N latitude,  in  the  vicinity  of  the  mean 
August  Tropical  Upper  Tropospheric  Trough  (TUTT) , which  were  more  erratic 
with  predominantly  northerly  components.  Both  of  these  concepts  support 
smaller  forecast  errors  for  storms  in  lower  latitudes. 
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Mean  forecast  errors  decrease  with  decreasing  east  longitude  (Figure 
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lb),  or  more  westerly  positions.  Generally,  a forecast  for  a storm  in 
a more  westerly  position  is  one  based  on  a longer  than  average  history, 
perhaps  in  an  area  of  better  synoptic  data  coverage,  given  the  proximity 
to  the  Philippines,  Taiwan,  and  China,  and  other  continental  areas  west 
of  130°E.  Additionally,  land  radar  enhances  accuracy  of  location. 

Storms  west  of  130°E  are  less  susceptible  to  large  forecast  errors  when 
land  is  nearby  to  the  north  of  the  track,  because  storms  unexpectedly 
recurving  over  China  dissipate  rapidly  and  hence  are  not  generally 
reflected  in  forecast  verification. 

Maximum  wind  (Figure  2)  is  another  important  parameter.  The  mean 
errors  decrease  with  increasing  maximum  wind  speeds,  indicating  that 
better  developed  storms,  again  with  longer  histories  and  more  accurate 
center  locations,  are  more  accurately  forecast.  This  trend  is  visible 
for  all  three  forecast  times,  with  some  increasing  fluctuations  and 
irregularities  in  the  48-,  and  72-hour  forecasts,  which  are  based  on 
progressively  smaller  sample  sizes. 

Relative  to  the  U component  of  storm  movement  (Figure  3a) , forecasts 
jure  generally  better  for  storms  moving  west  and  becoming  progressively 
more  difficult  as  westward  movement  diminishes  and  becomes  eastward  as 
associated  with  recurvature.  For  the  V component  (Figure  3b),  the  best 
forecasts  are  centered  at  or  near  zero,  again  implying  better  forecasts 
when  the  storm  is  moving  west  with  little  or  no  deflection  north  or  south. 

Errors  increased  markedly  for  storms  moving  south  with  any  component, 
or  to  the  north,  as  might  be  associated  with  the  recurvature  process. 

Time  of  day  (Figure  4b)  showed  no  perceivable  relationship  with  fore- 
cast errors  at  any  of  the  forecast  times.  For  the  years  1969-1971 

4 
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forecasts  were  issued  at  0500  GMT  plus  every  six  hours,  while  in  the 
remaining  years  forecasts  were  issued  at  0000  GMT  plus  every  six  hours. 
For  no  obvious  reason,  forecasts  in  those  three  years  appear  to  have 
been  superior  to  forecasts  issued  at  the  more  normal  synoptic  times. 

Time  of  year,  or  stratification  by  month  (Figure  4a) , did  show  that 
larger  errors  occurred  with  the  largest  freqeuncy  of  storms  in  late 
summer  to  fall.  There  is  a consistent  improvement  in  April  for  all  three 
forecasts,  but  since  this  is  based  on  only  5%  of  the  cases,  its  signi- 
ficance is  somewhat  dubious.  The  factors  of  workload  and  personnel  turn- 
over seem  to  be  reflected  in  Figure  4a.  Most  personnel  changes  occur  in 
the  spring  to  early  summer  months  as  the  frequency  of  multiple  storms 
increase.  Mean  errors  subsequently  increase  by  20  to  30%  in  July  and 
August,  and  then  taper  off  through  the  rest  of  the  season,  as  the  work- 
load stabilizes  and  as  the  newcomers  gain  experience.  This  trend  is 
less  pronounced  in  the  48-  and  72-hour  data,  but  in  light  of  the  case 
distribution,  the  argument  is  not  negated.  Support  for  this  argument  is 
shown  by  mean  errors  increasing  with  the  number  of  storms  occurring 
simultaneously  (Figure  5a) . This  could  be  indicative  of  the  aforemen- 
tioned added  workload  on  the  forecasters,  or  perhaps  due  to  complicated 
multistorm  interaction  not  fully  understood.  With  the  progressively 
fewer  number  of  cases  considered  for  an  increasing  number  of  storms, 
the  trend  is  not  strongly  supported,  however.  It  is  noted  that  four 
storm  cases  occurred  in  1972  only  and  may  reflect  the  year  rather  than 
the  occurrence  of  four  storms.  The  relationship  is  reinforced,  however, 
in  light  of  the  larger  errors  occurring  when  less  than  600  nautical  miles 
separates  two  storms.  This  parameter,  the  distance  to  the  nearest  storm, 
is  depicted  in  Figure  5b.  It  is  in  agreement  with  findings  by  Brand 
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(1968)  that  the  Fujiwhara  effect  is  not  felt  beyond  750  nautical  miles 


Beyond  that  distance,  mean  errors  decreased  and  stabilized.  The  para- 
meter of  six-hourly  point  along  the  track  (Figure  6a) , a measure  of  the 
length  of  storm  history,  showed  a trend  congruous  with  that  of  maximum 
wind:  as  the  storm's  history  and  development  increased,  the  forecast 
errors  decreased.  In  this  case,  a minimum  mean  error  occurs  late  in  the 
third  day  of  a storm.  This  stabilizes  in  the  24-hour  situation,  but 
decays  for  48-  and  72-hour  forecasts,  as  might  be  expected  with  storm 
recurvature  occurring  late  in  the  storm  history. 

For  the  last  variable,  initial  position  error  (Figure  6b) , a consis- 
tent and  prominent  trend  shows  the  mean  forecast  errors  increasing  as 
the  initial  position  errors  increase.  This  supports  the  basic  forecast- 
ing premise  that  accurate  observations  are  necessary  for  accurate  fore- 
casts. This  finding  is  in  general  agreement  with  that  of  Neumann  (1975), 
who  found  that  for  Atlantic  hurricanes,  the  initial  position  error  was 
important  in  objective  forecasts  with  its  relative  importance  decreasing 
in  longer-range  forecasts. 

C.  ANNUAL  VARIATION  OF  ERRORS 

Figure  7 shows  the  mean  forecast  errors  for  each  year  of  the  10  year 
sample,  with  the  least  squares  linear  trend  lines.  In  each  case,  the 
trend  line  is  too  shallow  to  indicate  conclusively  an  improvement  during 
the  10  years.  Correlations  are  negative  but  less  in  magnitude  than  0.3 
(about  one  standard  deviation  from  0.0).  While  the  hypothesis  that  there 
has  been  no  improvement  over  the  10  years  is  suspected,  it  cannot  be 
rejected.  Because  these  main  error  values  were  computed  including  all 
tropical  cyclones,  they  differ  from  those  published  in  the  annual  typhoon 
reports,  which  were  based  only  on  storms  of  typhoon  intensity. 
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D.  AUTOCORRELATION  OF  SUCCESSIVE  FORECASTS 


To  this  point,  the  forecasts  (and  hence  forecast  errors)  have  been 
tacitly  assumed  to  be  independent  of  each  other.  In  reality,  successive 
six-hourly  forecasts  for  a particular  storm  are  strongly  correlated. 

Table  III  gives  the  estimated  autocorrelation  coefficients  between  errors 
from  successive  forecasts  with  lag  times  out  to  36  hours.  It  is  possible 
to  adjust  the  number  of  related  cases  in  a particular  storm  downward  to 
an  effective  number  of  independent  cases  by  a complicated  relationship 
given  by  van  der  Bijl  (1951) . The  ratio  of  these  two  values  (the  effec- 
tive number  of  independent  cases  divided  by  the  total  cases)  decreases 
with  an  increase  in  the  autocorrelation  coefficient  as  well  as  with  in- 
creasing numbers  of  forecasts  per  storm,  and  increases  with  lag  time 
between  forecasts.  For  24-hour  forecasts,  where  typically  15  to  20  fore- 
casts are  made  at  six-hourly  intervals,  this  ratio  is  approximately  1/3. 

In  the  48-hour  case  where  the  autocorrelation  is  higher,  but  the  typical 
number  of  forecasts  per  storm  is  lower  (10-15) , this  ratio  is  1/4  to 
1/3.  At  72  hours,  the  six-hourly  autocorrelation  coefficient  is  higher; 
however,  the  forecasts  were  usually  issued  at  12-hourly  intervals  and 
the  typical  number  of  forecasts  was  about  five  per  storm,  thus  increasing 
the  ratio  to  1/3  to  1/2. 

This  ratio  is  important  in  significance  testing  where  the  square  root 
of  the  number  of  cases  (to  be  replaced  by  the  effective  number  of  indepen- 
dent cases)  is  found  in  the  denominator  of  the  test  statistic.  Whether 
1/4,  1/3,  or  1/2  of  the  number  of  cases  is  used  as  the  effective  number 
makes  little  difference  in  the  test  statistic,  so  1/3  times  the  number  of 
. cases  will  be  used  as  an  arbitrary  compromise  estimate  of  the  effective 

number  of  independent  cases  throughout  for  the  purpose  of  significance 
tepting. 
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E.  FREQUENCY  DISTRIBUTION  OF  FORECAST  ERROR  COMPONENTS 

For  the  purpose  of  constructing  probability  ellipses,  error  components 
have  been  assumed  to  be  distributed  according  to  a Guassian,  or  normal, 
frequency  distribution.  Figures  8a  and  8b  show  the  observed  cumulative 
frequency  distributions  of  the  West-East  (U)  and  South-North  (V)  components 
of  the  errors  plotted  on  probability  scaled  paper,  where  a normal  distri- 
bution would  be  represented  by  a straight  line.  This  presentation  shows 
generally  good  agreement  between  the  plotted  observed  and  normal  curves 
(straight  lines)  computed  from  estimates  of  the  means  and  standard  devia- 
tions. The  maximum  differences  between  the  theoretical  and  observed 
cumulative  frequencies  are: 


U COMPONENT 

V COMPONENT 

24HR 

4.5% 

errors  < - 70 

NMI 

3.9% 

errors  < - 70 

NMI 

48HR 

2.3% 

errors  < - 70 

NMI 

3.9% 

errors  < - 30 

NMI 

72HR 

2.2% 

errors  < -110 

NMI 

5.1% 

errors  < - 30 

NMI 

The  Kolmogorov-Smirnov  goodness  of  fit  test  (Massey,  1951)  regards 
as  significant  at  the  5%  level,  differences  in  observed  and  theoretical 
cumulative  frequencies  greater  than  1.36/v$J  in  absolute  value.  Using 
an  effective  number  of  cases  of  1/3  (4809)  = 1603  at  24  hours,  1/3  (3038) 

■ 1013  at  48  hours,  and  1/3  (1372)  * 457  at  72  hours;  the  cutoff  points 
for  significance  would  be  3.4%  at  24  hours,  4.3%  at  48  hours,  and  6.4% 
at  72  hours.  Only  the  differences  between  the  24-hour  theoretical  curves 
and  the  observed  plotted  values  are  significant  at  the  5%  level.  Esti- 
mates of  the  third  and  fourth  moments  about  the  mean  (Table  IV)  reveal 
that  the  24-hour  forecast  errors  are  skewed  west  (forecasts  are  too  far 
east)  while  all  other  skewness  coefficients  appear  normal.  Both  components. 
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however,  appear  to  be  leptokurtic.  This  is  evident  in  Figure  8,  where 
extreme  occurrences  fall  counterclockwise  with  respect  to  the  theoretical 
lines.  This  suggests  that  probability  ellipses,  based  on  the  assumption 
of  Gaussian  distributions,  may  be  slightly  biased  in  that  24-hour  verify- 
ing positions  are  more  likely  to  fall  out  of  an  ellipse  on  the  west  side 
that  the  east  side,  and  that  inner  and  outer  ellipses  may  not  contain  the 
proper  proportion  of  the  verifying  positions.  In  general  the  10%  ellipses 
would  be  expected  to  contain  more  than  10%  of  the  verifying  positions  and 
the  area  beyond  the  95%  ellipse  to  contain  more  than  5%  of  the  verifying 
positions.  It  is  not  possible  to  make  a statement  about  the  intermediate 
ellipses  between  10  and  95%,  but  Figure  8 suggests  good  agreement  between 
the  theoretical  and  observed  cumulative  frequencies  there. 

If  probability  ellipses,  or  o-.her  estimates  of  future  probable  error, 
integrated  over  an  area  are  desired,  the  observed  cumulative  distribution 
could  be  used  in  place  of  the  Gaussian  cumulative  distribution.  The 
degree  of  complexity  added  by  such  a step,  as  well  as  the  uncertainty  in 
the  representativeness  of  this  particular  10  year  sample  (to  the  future) 
suggest  such  a step  is  not  warranted. 
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IV.  DISCRIMINANT  ANALYSIS 


A.  CALCULATION  OP  DISCRIMINANT  FUNCTIONS  AND  GROUP  MEANS 

Seeking  to  identify  forecasts  as  either  good  or  bad,  discriminant 
analysis  was  used,  namely,  the  UCLA  Biomedical  computer  program  BMDP7M 
(Dixon,  1974) . With  this  approach,  the  cases  were  divided  into  groups 
and  classification  functions  were  found  that  best  delineate  the  groups. 
These  functions,  linear  combinations  of  the  variables,  would  then  be 
used  to  predict  the  classification  of  new  cases.  BMDP7M  is  the  stepwise 
discriminant  analysis  which  identifies  the  subset  of  variables  that 
maximizes  the  difference  between  groups.  Variables  are  entered  into 
the  classification  function  one  at  a time  until  there  is  no  appreciable 
improvement  in  group  separation. 

For  this  study,  U and  V error  components  were  either  good,  with  the 
absolute  values  of  the  error  less  than  or  equal  to  the  median;  or  bad, 
with  the  absolute  value  of  the  error  greater  than  the  median.  The  four 
possible  combinations  were  resolved  into  three  classifications: 

GROUP  1:  both  U and  V components  good 

GROUP  2:  either  U or  V good 

GROUP  3:  both  U and  V components  bad 

The  six  variables  contributing  to  the  separation  of  groups  included  all 
those  selected  by  the  linear  regression: 


1. 

Latitude 

LAT 

2. 

Longitude 

LONG 

3. 

Maximum  Wind 

MAX  WND 

4. 

West-East  Component  of  Movement 

U MOVT 
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5.  South-North  Component  of  Movement 


V MOVT 


6.  Number  of  Storms  in  Progress  at  Forecast  Time  NO  STM 
Their  means  and  standard  deviations  are  given  in  Table  V. 

Previously  established  trends  are  consistently  apparent  in  the  data. 
Group  1 forecasts  are  associated  with  lower  latitudes,  more  westerly 
longitudes,  faster  westerly  movement,  minimal  N-S  movement,  the  more 
intense  storms  (typhoons),  and  a fewer  number  of  concurrent  storms. 

B.  TESTING  OF  THE  DISCRIMINANT  FUNCTIONS  AND  GROUP  MEANS 

Each  set  of  forecast  cases  was  then  tested  by  applying  the  group 
means  and  classification  functions  calculated  from  the  BMDP7M  program, 
using  the  24  hour  data.  The  classification  functions  derived  from  the 
24-hour  forecasts  were  also  applied  to  48-  and  72-hour  forecasts.  This 
resulted  in  slight  loss  of  discrimination  at  48  hours,  but  has  the 
advantage  of  classifying  a forecast  into  the  same  category  for  forecasts 
at  all  time  intervals. 

Using  the  six  resulting  coefficients  (c. ,c.)  and  a constant 

1 O 

(c_)  for  each  group,  operating  on  the  six  selected  variables  (x, , . . . . ,x.) , 
three  functions  (f)  were  evaluated  for  each  group  thuslys 


CliXl  + C2iX2 


• # • 


c^.x.  + c_ . 
6i  6 7i 


i=  1,2,3 


Each  function  value  was  subtracted  from  its  corresponding  component  of 
the  g*.oup  means,  and  the  differences  were  squared  and  summed  to  represent 
a vector  distance  from  each  group  mean.  Each  case  was  then  assigned  to 
one  of  three  groups  according  to  which  vector  distance  was  minimal.  The 
cases  so  sorted  were  counted,  and  the  means  and  standard  deviations  of 
the  error  components  and  error  magnitudes  were  calculated.  These  means 
and  standard  deviations  are  given  in  Table  IV.  The  standard  deviations 
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of  the  components  and  the  mean  vector  errors  reflect  the  differences  in 
the  groups,  with  the  component  means  mostly  near  zero.  A Student's  t 
test  was  applied  to  determine  if  the  mean  vector  errors  of  each  group 
were  significantly  different  from  those  of  the  other  groups.  At  the  5% 
level  of  significance  for  a one  tail  test,  t = 1.645.  Therefore,  if 
the  value  of  t between  two  groups  exceeds  that  figure,  the  groups  are 
deemed  to  be  significantly  different.  Values  of  t computed  on  the  group 
means  are  given  in  Table  VII.  It  should  be  noted  that  the  number  of  cases 
was  reduced  by  a factor  of  3 to  account  for  autocorrelation  as  previously 
discussed. 

The  mean  errors  of  Group  1 were  found  to  be  significantly  less  than 
those  of  Group  2,  and  those  of  Group  2 were  significantly  less  than  those 
of  Group  3,  except  for  the  72-hour  data.  There  the  mean  V components  of 
the  errors  were  significantly  non-zero  and  of  different  signs,  giving 
unique  spatial  error  distributions  for  the  two  groups  with  only  slightly 
different  mean  absolute  errors. 

C.  GROUP  ANNUAL  VARIATIONS 

Mean  forecasts  errors  per  year  per  group,  and  all  groups  combined,  are 
shown  in  Figure  9 with  trend  lines.  Again  the  difference  between  Group  1 
and  3 is  substantial.  The  Group  2 average  errors  most  closely  approximate 
the  pattern  of  all  three  groups  combined.  Year  to  year  fluctuations  are 
extremely  large  for  Group  2,  and  somewhat  less  for  Groups  1 and  3.  The 
larger  fluctuations  in  the  Group  2 means  and  in  the  mean  errors  of  longer- 

I 

• range  forecasts  in  all  groups  is  mostly  attributable  to  relatively 

smaller  sample  sizes. 
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D.  PROBABILITY  ELLIPSE  COMPUTATIONS 

Having  thus  far  separated  forecasts  into  one  of  three  groups,  it  was 
now  desirable  to  present  this  information  in  a more  graphical  way;  the 
goal  being  a useful  operational  application.  Following  previously  estab- 
lished methods  (Stevens  and  Palmer,  1963) , the  probability  ellipse  is 
such  an  application.  Assume  that  errors  in  forecast  position  approximate 
a bivariate  normal  distribution  (Section  III.E) . The  expression  for  an 
ellipse  is  given  by: 

2 2 2 2 

x - 2rxy  + y = (1  - r)  c , 

with  the  normalized  error  components  x = (U-U)/su  , and  y = ( V-V)/s . 

U is  the  E-W  error  component;  V is  the  N-S  error  component.  U,  V,  su, 

and  sv  are  the  estimates  of  the  respective  means  and  standard  deviations; 

and  r is  the  estimated  correaltion  coefficient  between  U and  V. 

-c2/2 

Probability  = 1 - e , c = 1 approximates  a 40%  ellipse.  Figure  10 
shows  the  40%  probability  ellipses  for  each  group  at  each  forecast  interval. 
Distance  dimensions  are  nautical  miles,  areas  are  in  thousands  of  square 
nautical  miles,  and  directions  are  in  degrees  north  of  east.  A 40% 
probability  ellipse  means  that  a forecast  position  has  a 40%  probability 
of  falling  within  its  corresponding  ellipse.  Only  the  difference  in  size 
between  Groups  1 and  3 is  immediately  obvious,  but  upon  comparison  of  the 
areas,  the  distinctions  are  more  pronounced.  The  24-hour  area  of  Group  3 
is  more  than  double  that  of  Group  1,  with  the  48-  and  72-hour  areas  being 
97  and  57%  larger,  respectively. 

The  general  NW  orientation  of  the  major  axis  of  the  ellipse  indicates 
that  for  low  latitude  3torms  on  a normal  WNW  track,  errors  are  comprised 
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of  nearly  equal  components  along  the  track  (speed  error) , and  across  the 


track  (track  error) . These  storms  are  usually  associated  with  Groups  1 
and  2.  Recurving  and  post-recurvature  storms  are  almost  always  in  Groups 
2 and  3,  with  Group  3 predominating  as  storms  are  entrapped  by  the 
westerlies.  During  recurvature,  when  the  track  is  nearly  north,  track 
error  is  dominant,  whereas  after  recurvature,  speed  error  dominates. 

Using  too  large  an  ellipse  (such  as  Group  2 for  a Group  1 case)  tends 
to  dilute  and  spread  the  estimated  probability  density.  This  has  the 
effect  of  overwarning  those  customers  far  removed  from  the  forecast  track, 
and  underwarning  those  along  the  track.  Conversely,  using  too  small  an 
ellipse  (such  as  Group  2 for  a Group  3 case)  has  the  effect  of  overwarn- 
ing those  along  the  forecast  track  and  underwarning  those  in  the  periphery. 
This  case  is  the  meteorologist's  familiar  dilemma  when  forecasts  are 
taken  too  literally  without  adequate  allowance  for  errors.  Tailoring 
the  ellipses  to  the  expected  forecasting  difficulty  has  the  effect  of 
reducing  both  overwarning  and  underwarning. 
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TESTING  OF  INDEPENDENT  DATA  (1976) 


The  final  step  in  this  research  was  to  apply  the  same  procedures  to 
an  independent  data  set  and  compare  the  results. 

A.  DISTRIBUTION  OF  ERRORS 

The  1976  forecasts  and  best  tracks,  having  been  processed  like  the 
10  year  sample,  were  analyzed  using  the  same  discriminant  functions  and 
group  means  to  similarly  arrive  at  three  separated  groups,  the  1976  group 
statistics  are  listed  in  Table  VIII.  The  contrast  between  group  statis- 
tics is  not  as  sharp  as  in  the  10  year  independent  data  sample  (Table  VI) . 
The  means  vary  more  widely  as  compared  to  the  dependent  data,  while  the 
smaller  standard  deviations  of  Group  2 show  the  1976  forecasts  to  appear 
significantly  better  for  those  cases. 

So  the  question  arises  as  to  whether  errors  of  1976  are  representa- 
tive of  the  10  year  data  sample.  Statistically  each  group  vector  mean 
of  the  1976  data  was  compared  with  its  counterpart  in  the  dependent  data 
sample.  Table  IX  lists  the  values  of  the  normal  test  statistic,  Z, 
found  for  each  group  to  be  compared  at  the  5%  level  of  significance  value 
of  Zs  1.96.  From  this.  Groups  1 and  3 of  1976  cannot  be  rejected  as 
having  come  from  the  sample  of  the  previous  10  years,  but  Group  2 fore- 
casts appear  to  be  significantly  better  than  the  preceding  10  year  average. 

On  the  other  hand.  Figure  9 shows  Group  2 to  have  a wide  annual  varia- 
tion, partially  because  of  smaller  frequency  of  occurrence.  The  relative- 
ly high  number  of  Group  2 cases  in  1976  may  include  only  a few  storms 

which  can  negate  any  significance  in  the  above  differences. 
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B.  ELLIPSE  TESTING 

Forecast  errors  of  1976  were  tested  to  determine  the  percentages  of 
verifying  positions  that  would  fall  within  ellipses  with  probabilities 
specified  at  25,  50,  75,  90,  and  95%.  These  results  appear  in  Figure  11. 
For  Group  1,  the  observed  closely  follows  the  45°  expected  line  with  the 
maximum  deviation  of  observed  from  expected  being  10%  at  72  hours. 

Group  2 deviations  were  consistently  conservative  (above  the  45°  line) 
with  deviations  up  to  18%.  For  Group  3 the  72 -hour  deviation  was  the 
greatest  at  20%,  also  conservative.  None  of  these  differences  are  signi- 
ficant at  the  5%  level  by  the  Kolmogorov-Smirnov  goodness  of  fit  test. 
Generally,  the  ellipses  were  too  conservative.  This  is  to  be  expected 
since  a trend  of  decreasing  errors  was  evident  in  the  10  year  data. 

For  dramatic  comparison,  the  same  ellipse  testing  was  repeated,  but 
counting  the  number  of  Group  3 cases  that  fell  into  the  smaller  Group  1 
ellipses  and  the  number  of  Group  1 cases  that  fell  into  the  larger  Group  3 
ellipses  (Figure  12).  The  comparison  shows  the  Group  3 ellipses  to  be 
ultra-conservative  for  Group  1 cases.  Conversely,  fewer  Group  3 observed 
cases  fell  into  Group  1 ellipses,  by  roughly  the  same  percentages  below 
the  expected  as  the  other  situation  was  above.  This  contrast  shows 
significant  differences  exist  in  the  distribution  of  errors  from  fore- 
casts specified  in  advance  to  be  either  Gro\:p  1 or  Group  3. 
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VI.  CONCLUSIONS 


In  light  of  the  objectives  outlined  for  this  study,  to  some  extent 
the  long-range  goals  have  been  attained.  It  has  been  demonstrated  that 
a small  number  of  readily  available  parameters  can,  with  reasonable 
effectiveness,  classify  a tropical  cyclone  forecast  as  likely  resulting 
in  either  markedly  above  or  below  average  errors.  Group  1 forecasts 
have  a high  probability  of  below  average  errors  with  a low  probability 
of  above  average  errors.  Group  2 forecasts  have  approximately  equal 
probabilities  of  being  above  or  below  average.  Group  3 forecasts  have 
a low  probability  of  below  average  errors  with  a high  probability  of 
above  average  errors. 

The  concept  of  using  least  squares  regression  to  predict  in  advance 
the  actual  error  (as  opposed  to  a class  of  errors)  appears  to  offer 
little  chance  of  meaningful  success.  It  is  apparent  that  it  is  possible 
to  isolate  conditions  contributing  to  forecast  errors  in  the  mean,  but 
one  must  bear  in  mind  that  excellent  forecasts  are  occasionally  made 
under  the  worst  conditions,  and  conversely,  terrible  forecasts  can  be 
made  under  the  best  of  conditions. 
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VII.  RECOMIENDAT IONS 

The  examination  of  two  additional  parameters  could  improve  the 
delineation  between  classifications.  First,  initial  position  error, 
shown  to  be  directly  related  to  the  forecast  errors,  was  not  intro- 
duced as  a discriminator  because  it  is  not  generally  known  to  the 
forecaster  at  the  time  of  the  forecast. 

Second,  the  synoptic  pattern  associated  with  each  storm  has  not 
been  considered.  Some  parameter  which  accounts  for  the  relative 
locations  of  semi-permanent  features;  such  as  the  TUTT,  subtropical 
ridges,  and  perhaps  transient  troughs  in  the  westerlies;  might  prove 
to  be  a most  inqportant  discriminator,  especially  as  it  relates  to  the 
track  forecast  errors  and  the  basic  problem  of  forecasting  the 
recurvature  of  tropical  cyclones. 

While  these  results  are  to  be  considered  preliminary,  pending 
improvements  and  refinements,  dissemination  to  typhoon  forecast  sub- 
scribers is  recommended. 


TABLE  I.  BASIC  STATISTICS  - RESULTS  OF 

STEPWISE  MULTIPLE  LINEAR  REGRESSIONS 


VARIABLE 

MEANS 

STANDARD 

DEVIATION 

VARIANCE  EXPLAINED 

(%) 

24-HR 

48-HR 

72-HR 

MAX  WND  (kts) 

65.9 

28.8 

3.4 

1.4 

0.4 

LAT  ( °N) 

18.7 

6.3 

4.2 

1.1 

0.6 

LONG  ( °E) 

135.0 

14.6 

1.8 

2.3 

2.3 

U MOVT  (kts) 

-5.5 

6.4 

0.3 

3.3 

2.9 

V MOVT  (kts) 

4.0 

4.1 

0.5 

0.5 

0.3 

POS.  NO. 

13.5 

10.6 

* 

NE 

NE 

NE 

NO.  STM. 

1.5 

0.7 

0.3 

0.2 

NE 

MONTH 

8.7 

2.3 

NE 

NE 

0.1 

TIME  GMT  (hrs) 

10.3 

7.0 

NE 

NE 

NE 

ERR  DIS:  24-HR 
(NMI ) 

125.7 

80.8 

ERR  DIS:  48-HR 
(NMI) 

247.0 

153.8 

ERR  DIS:  72-HR 
(NMI) 

369.4 

226.0 

TOTAL  VARIANCE 

EXPLAINED 

(%) 

10.5 

8.8 

6.6 

* 

NE:  Was  not  entered  in  linear  regression 
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TABLE  III.  AUTOCORRELATIONS  BETWEEN  FORECAST 
ERRORS  OF  SUCCESSIVE  FORECASTS 


TIME  LAG  (HOURS) 

24- HR 

48- HR 

7 2- HR 

0 

1.000 

1.000 

1.000 

6 

.665 

.790 

.838 

12 

.432 

.587 

.675 

18 

.291 

.432 

.476 

24 

.213 

.305 

. 371 

30 

.173 

.212 

.127 

36 

.181 

.171 

.177 

TABLE  IV. 

SKEWNESS  AND  KURTOSIS 
OF  ERROR  COMPONENTS 

COEFFICIENTS 

U 

NORMAL 

24-HR 

4 8 -HR 

7 2 -HR 

3rd  Moment 
Skewness  Coef. 

0.0 

-1.58* 

-.013 

.046 

4th  Moment 
Kurtosis 

3.0 

4.434 

3.753* 

3.522* 

V 

3rd  Moment 

0.0 

0.18 

-.061 

.067 

4th  Moment 

3.0 

4.544* 

3.881* 

- 3.185 

* 

Significantly 

different 

from  the  Gaussian  values. 
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TABLE  VI.  TEST  GROUP  STATISTICS 


FORECAST 

MEANS 

STD. 

DEV. 

MAGNITUDE  OF  VECTOR  ERROR 

24-HR 

GROUP 

CASES 

U 

V 

U 

V 

MEAN 

STD.  DEV. 

1 

1834 

-6.0 

3.1 

87.9 

75.3 

98.8 

57.9 

2 

1310 

-16.0 

-5.1 

119.4 

95.9 

130.4 

84.0 

3 

1665 

8.5 

-3.1 

129.6 

118.4 

151.1 

89.8 

48-HR 

1 

1432 

-15.0 

4.1 

195.9 

151.4 

211.1 

130.3 

2 

733 

-19.4 

-31.4 

237.0 

179.8 

242.2 

181.3 

3 

873 

19.2 

15.6 

258.7 

223.0 

295.7 

172.7 

7 2 -HR 

1 

623 

-14.4 

2.2 

309.5 

235.4 

326.0 

212.5 

2 

367 

-8.1 

-70.5 

370.2 

249.3 

391.2 

238.0 

3 

382 

17.0 

27.5 

368.4 

302.5 

418.5 

230.6 

TABLE  VII.  STUDENT'S  t VALUES  COMPUTED  ON 
GROUP  MEANS  (10  YEAR  DATA) 


24-HR 

48-HR 

72 -HR 

Group 

1 vs.  2 

7.2 

2.65 

2.56 

Group 

2 vs.  3 

3.7 

3.48 

0.92 
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TABLE  VIII 


1976  TEST  GROUP  STATISTICS 


FORECAST 

MEANS 

STD. 

DEV. 

MAGNITUDE 

OF  VECTOR  ERROR 

24-HR 

GROUP 

CASES 

U 

V 

U 

V 

MEAN 

STD.  DEV. 

1 

144 

-30.6 

-15.9 

84.8 

80.6 

104.6 

62.6 

2 

240 

-25.0 

3.8 

96.1 

75.4 

110.4 

58.2 

3 

140 

5.5 

9.3 

111.9 

122.4 

143.3 

84.2 

48-HR 

1 

133 

-75.1 

-24.0 

196.5 

160.1 

226.9 

137.8 

2 

202 

-57.6 

13.2 

192.3 

145.9 

216.9 

121.4 

3 

89 

42.2 

33.4 

214.7 

195.6 

265.2 

130.0 

72 -HR 

1 

109 

-110.8 

-35.2 

328.0 

240.1 

357.3 

225.9 

2 

164 

-72.1 

11.2 

289.5 

206.3 

312.4 

184.7 

3 

59 

69.2 

65.0 

337.1 

204.5 

376.0 

151.9 

TABLE  IX.  NORMALIZED  DEVIATIONS  OF  1976 

GROUP  MEANS  FROM  10  YEAR  GROUP  MEANS 


GROUP 

24-HR 

48-HR 

7 2 -HR 

1 

0.60 

0.80 

0.88 

2 

-2.12 

-1.15 

-2.45 

3 

-0.51 

-0.96 

-0.82 
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MEAN  FORECAST  ERROR 


MEAN  FORECAST  ERROR 


U MOVT  (kts) 


V MOVT  (kts) 


Figure  3.  Mean  Error  Stratifications  by  West-East,  South 
North  Components  of  Movement. 


MEAN  FORECAST  ERROR 


MEAN  ERROR  STRATIFICATIONS 


Figure  4.  Mean  Error  Stratifications  by  Month,  Time  (GMT) 


MEAN  FORECAST  ERROR 


MEAN  ERROR  STRATIFICATIONS 
Fig.  5a.  Number  of  Storms  5b.  Distance  to  Nearest  Storm 

N Ml 

430-  NUMBER  OF  CASES  PER  STRATIFICATION  SHOWN  WHEN  LESS  THAN  100 


48 

HOUR 


24 

HOUR 


24  HOUR  FREQUENCY  DISTRIBUTION  ( % ) 


NO  STM 


STM  DIS  (NMI) 


Figure  5.  Mean  Error  Stratifications  by  Number  of  storms 
Distance  to  the  Nearest  Storm. 


rigure  6.  Mean  Error  Stratifications  by  Point  on  Track 
Initial  Position  Error. 
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MEAN  ERROR  STRATIFICATIONS 


Figure  7.  Mean  Error  Stratifications  by  Year. 
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CUMULATIVE  FREQUENCY  (X) 


6 66 
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Figure  8b.  Error  Frequency  Distribution  - V Component. 
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Figure  10.  40%  Probability  Ellipses  Per  Group  Per  Forecast. 
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OBSERVED  CASES  («) 
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OBSERVED  CASES  (%) 


§ 


2 

(a)  GROUP  1 

CASES  IN  GROUP  3 ELLIPSES 

24  Hour 

48  Hour 

72  Hour 

23  SO  73  100 


EXPECTED  CASES  (%) 

| t MAXIMUM  DEVIATION  (OBSERVED  TO  EXPECTED) 


Figure  12.  contrast  of  Group  1 and  Group  3 1976  Verifying 
Positions  that  Fell  into  25/50,75,90,  and  95% 
Probability  Ellipses  for  Group  1 and  Group  3 by 
Forecast  (24,  48,  and  72  hours). 
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